The Generalised Discrete Algebraic Riccati Equation 

in LQ optimal control * 



Augusto Ferrante* Lorenzo Ntogramatzidis* 

*Dipartimento di Ingegneria dell' Informazione 
Universita di Padova, via Gradenigo, 6/B - 35131 Padova, Italy 
augustoSdei . unipd . it 
^Department of Mathematics and Statistics 
Curtin University, Perth WA, Austraha. 
L . NtogramatzidisOcurtin . edu . au 

January 19, 2012 



Abstract 

This paper investigates the properties of the solutions of the generalised discrete 
algebraic Riccati equation arising from the solution of the classic infinite-horizon 
linear quadratic control problem. In particular, a geometric analysis is used to study 
the relationship existing between the solutions of the generalised Riccati equation 
and the output-nulling subspaces of the underlying system and the corresponding 
reachability subspaces. This analysis reveals the presence of a subspace that plays 
an important role in the solution of the related optimal control problem, which is 
reflected in the generalised eigenstructure of the corresponding extended symplectic 
pencil. 

In establishing the main results of this paper, several ancillay problems on the discrete 
Lyapunov equation and spectral factorisation are also addressed and solved. 
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1 Introduction 

Ever since in the early sixties Kalman described in his pioneering papers [fTTl [T2ll the crucial role of 
Riccati equations in the solution of the linear quadratic (LQ) optimal control and filtering problems, 
the range of control and estimation problems where Riccati equations have been discovered to play 
a fundamental role has been increasing dramatically. Indeed, in the last fifty years Riccati equations 
have been found to arise also in linear dynamic games with quadratic cost criteria, spectral factorisation 
problems, singular perturbation theory, stochastic realization theory and identification, boundary value 
problems for ordinary differential equations, invariant embedding and scattering theory. For this reason, 
Riccati equations are universally regarded as a cornerstone of modem control theory. Several monographs 
have been entirely devoted to providing a general and systematic framework for the study of Riccati 
equations, see e.g. ll20l[T3i|9l[Tl. 

In the continuous time, the structure of the solution of a linear-quadratic problem strongly depends 
on the rank of the matrix penalising the control in the performance index, which is traditionally denoted 
by R. When R is non- singular the optimal control can be found by solving a Riccati equation (which is 
differential or algebraic depending on the horizon of the performance index). Indeed, in this case, such 
Riccati equation - which explicitly involves the inverse of 7? - is well-defined. But when R is singular, 
a solution of the problem is guaranteed to exist for all initial conditions only if the class of allowable 
controls is extended to include distributions [|7l|2Tl[l5l, and the Riccati equation is not defined. 



In the discrete time, the classic solution of the infinite-horizon LQ problem is traditionally expressed 
in terms of the solution X of the Riccati equation 

X =a''xa-{a''xb+s){r+b^xb)-\b^xa+s^)+q, (1) 

where A e M"""', B e M"""", Q G M"""", S e M"""" and R e M'"""" are such that 

:n^>0. (2) 



Q s 

R 



Matrix 11 is usually referred to as Popov matrix. The set of matrices E — {A,B; Q,R,S) is often referred to 
as Popov triple, see e.g. [9]. Equation ([T]) is the so-called Discrete Riccati Algebraic Equation DARE(E). 
Notice that now it is not the inverse of R that explicitly appears in the Riccati equation but the inverse of 
the term R + B^XB, which can be non-singular even when R is singular. Nevertheless, even though the 
distinction between the cases in which R is invertible or singular needs not be considered, very often even 
in the discrete time it is assumed that R is non-singular because this assumption considerably simplifies 
several underlying mathematical derivations. 

However, even the solution to the infinite-horizon LQ problem expressed in terms of matrices satisfy- 
ing this equation is somehow restrictive. Indeed, an LQ problem may have solutions even if DARE has 
no solutions, and the optimal control can be written in this case as a state feedback written in terms of a 
matrix X such that R + B^ X B is singular and satisfies the more general Riccati equation 

X = A^XA-{A^XB+S){R+B^XB)\B^XA+S^)+Q, (3) 
ker{R + B^XB) Cker{A^XB + S), (4) 

where the matrix inverse in DARE(E) has been replaced by the Moore-Penrose pseudo-inverse, see 
[fT6l . Equation ([3]) is known in the literature as the generalised discrete-time algebraic Riccati equa- 
tion GDARE(E). The GDARE(E) with the additional constraint on its solutions given by dH) is some- 
times referred to as constrained generalised discrete-time algebraic Riccati equation, herein denoted by 
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CGDARE(E). It is obvious that ([3]) constitutes a generalisation of the classic DARE(E), in the sense 
that any solution of DARE(E) is also a solution of GDARE(E) - and therefore also of CGDARE(E) be- 
cause (SI) is automatically satisfied since ker{R + X B) = 0,n - but the vice-versa is not true in general. 
Despite its generality, this type of Riccati equation has not yet received a great deal of attention in the 
literature. It has only been marginally studied in the monographs [fTTl l9l [Til and in the paper [3|. The 
only comprehensive contributions entirely devoted to the study of the solutions of this equation are [[8]| 
and flS] . The former investigates conditions under which the GDARE(E) admits a stabilising solution 
in terms of the deflating subspaces of the extended symplectic pencil. The latter studies the connection 
between the solutions of this equation and the rank-minimising solutions of the so-called Riccati linear 
matrix inequality. In pursuing this task, the authors of IfTSlI derived a series of results that shed some 
light into the structural properties of the solutions of the generalised Riccati equation, and in particular 
in the fundamental role played by the term R + B^XB. An example is the important observation ac- 
cording to which the inertia of this matrix R + B^ X B - that from now on we will denote by Rx for the 
sake of conciseness - is independent of the particular solution X satisfying CGDARE(E), [18, Theorem 
2.4]. This implies that a given CGDARE(E) cannot have one solution X = such that R-\- B^XB is 
non-singular and another solution Y = Y^ for which R + B^Y B is singular. As such, i) if X is a solution 
of DARE(E), then aU solutions of CGDARE(E) will also satisfy DARE(E) and, ii) if X is a solution of 
CGDARE(E) such that R + B^ X B is singular, then DARE(E) does not admit solutions. The results pre- 
sented in [18] are established in the very general setting in which the Popov matrix IT is not necessarily 
positive semidefinite as in (O. 

In this paper we are interested in the connection of the use of the CGDARE(E) in the solution of 
optimal control or filtering problems - the so-called H2-DARE in the terminology of [[T8]| . It is often 
taken for granted that the generalised discrete-time Riccati equation generalises the standard DARE(E) 
in the solution of the infinite LQ optimal control problem in the same way in which [fT6l established 
that the generalised Riccati difference equation generalises the standard Riccati difference equation in 
the solution of the finite-horizon LQ problem. However, to the best of the authors' knowledge, this fact 
has never been presented in a direct, self-contained and rigorous way. Thus, the first aim of this paper is 
to fill this gap, by showing in an elementary, yet rigorous, way, the connection of the CGDARE(E) and 
the solution of the standard infinite-horizon LQ optimal control problem. The second aim of this paper 
is to provide a geometric picture describing the structure of the solutions of the CGDARE(E) in terms 
of the output nulling subspaces of the original system E and the corresponding reachability subspaces. 
Indeed, under the usual assumption of positive semidefiniteness of the Popov matrix, the null-space of 
Rx is independent of the solution X of CGDARE(E). Even more importantly, this null-space is linked 
to the presence of a subspace - that will be identified and characterised in this paper - which plays 
an important role in the characterisation of the solutions of CGDARE(E), and also in the solution of 
the related optimal control problem. This subspace does not depend on the particular solution X, nor 
does the closed-loop matrix restricted to this subspace. This new geometric analysis will reveal that the 
spectrum of the closed-loop system is divided into two parts: the first depends on the solution X of the 
CGDARE(E), while the second - coinciding exactly with the eigenvalues of the closed-loop restricted to 
this subspace - is independent of it and does not appear in the generalised eigenstructure of the extended 
symplectic pencil. At first sight, this fact seems to constitute a limitation in the design of the optimal 
feedback, because it means that regardless of the solution of the generalised Riccati equation chosen for 
the implementation of the optimal feedback, the closed-loop matrix will always present a certain fixed 
eigenstructure as part of its spectrum. However, when R + B^ X B is singular, the set of optimal controls 
presents a further degree of freedom - which is also identified in ifTTl Remark 4.2.3] - that allows to place 
all the poles of the closed-loop system at the desired locations without changing the cost. 

Several other important ancillary results of independent theoretical interest are derived in this paper. 
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These include interesting considerations on the solutions of Hermitian Stein equations and spectral 
factorisation results that generalise the classic ones in more than one direction. 



2 Linear Quadratic optimal control and CGDARE 

In this section we analyse the connections between Linear Quadratic (LQ) optimal control and CGDARE. 
Most of the results presented in this section are considered "common wisdom". However, we have not 
been able to find a place where they have been derived in detail, so we believe that this section may 
be useful. Consider the classic LQ optimal control problem. In particular, consider the discrete linear 
time-invariant system governed by 

Xt-^i=Axt+But, (5) 

where A G M"^" and 5 G M"^™, and let the initial state xq G be given. The problem is to find a sequence 
of inputs Ut, with t — 0,1,...,°°, minimising the cost function 



J{xQ,u)= [ xJ uJ ] 



' Q s' 






R 







(6) 



Before we introduce the solution of the optimal control problem, we recall some classic linear al- 
gebra results which will be useful in the sequel. We also give a proof of these results for the sake of 
completeness. 
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P,^. Pll 



Then, 



Lemma 2.1 Consider the symmetric positive semidefinite matrix P ■ 

(i) kerPi2 5 kerP22; 

(ii) Px2Pl2P22-Pn; 

(iii) Pi2(/-Pj2^22)=0; 

(iv) Pn-PnPl2Pn>^- 

Proof: (i) Since /' = i'^>0, two matrices C and D exist such that P= [C Dy[C D] so that P12 = C^D 
and P22—D^ D. Let x G ker /'22- ThQn,0—x^D^Dx~ | |Z)jc| p, which gives Z)x = 0. This in turn implies 
that xG ker P12. (ii) The inclusion kerPi2 2 ker/'22 can be rewritten as im P^2 Q im ^22- Thus, a matrix 
^gj^nxm exists such that P\2 = KP22- On post-multiplying both sides of this identity by ^22^22 we 
obtain ^12^22^22 = KP22P22P22 = ^^22 = Pi2- (iii) Since as already proved kerP22 Q kerPi2, a matrix 
K exists such that Pn = KP22. Therefore, Pn (I - PI2 P22) = KP22 (I - P22 P22) = K (P22 ^22 ^22 ^22) = 



K {P22 - P22) = 0. (iv) It follows directly from Pn - ^12^22^2 = -^12 



4] 



^'ii P12 



-p' pi 



>0. 



We now introduce some notation that will be used throughout the paper. First, to any matrix X =X^ e 



l"^" associate the following matrices: 

Q+A^XA-X, Sx 



Qx 
Gx 
Kx 

Ax 

Ux 



= A^XB+S, Rx = R+B^XB, 



— ^1 



4rx, 



{r+b''xb)\r+b^xb) = i„ 
= {r+b^xb)\b^xa+s^)=rIsI 

-B{R + B^XB)\B^XA + S^) ^A-BKx, 
Qx Sx 



def ■ 

= A 



SI 



Rx 



(7) 
(8) 
(9) 
(10) 

(11) 
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The term Rj^Rx is the orthogonal projector that projects onto imi?^ = imRx so that Gx is the orthogonal 
projector that projects onto kerRx- Hence, kerRx = imGx- When X is a solution of CGDARE(E), 
then Kx is the corresponding gain matrix, Ax the associated closed-loop matrix, and is the so-called 
dissipation matrix. It is easy to see that all symmetric and positive semidefinite solutions of GDARE(E) 
satisfy (01), and are therefore solutions of CGDARE(E). In fact, if X is positive semidefinite, we find 



Sx ' 






Rx 







X[A 5 ] +n>o. 



Therefore, applying Lemma [TTI we find (H)), that can be rewritten as keri?^ C kerSx and also as Sx Gx = 
0. 

The following fundamental result holds. 

Theorem 2.1 Suppose that for every xq there exists an input Ut G W^, with f G N, such that J{xq,u) is 
finite. Then we have: 

1. CGDARECL) admits symmetric solutions: A solution X = > may be obtained as the limit of 
the sequence of matrices generated by iterating the generalised Riccati difference equation ( see 
di?]) below) with zero initial condition. 

2. The value of the optimal cost is XqXxq. 

3. X is the minimum positive semidefinite solution of CGDARECL). 

4. The set o/all optimal controls minimising © can be parameterised as 

Ut = -KxXt + GxVt, (12) 

with arbitrary Vt. 

Proof: (1). Consider the finite horizon LQ problem consisting in the minimisation of the performance 
index with zero terminal cost 

T-l 



Ut 



(13) 



subject to Q with assigned initial state xq G W\ The optimal control is obtained (see e.g. [fT6ll ) by 
iterating, backward in time starting from the terminal condition Pt{T) = 0, the generalised Riccati 
difference equation /V(0 = + 1)]^ where R[-] is the Riccati operator defined as 

R[P] =A^PA-{A^PB+S){R+B^PBy{B^PA+S^)+Q (14) 

and the optimal value of the cost is 7f (xq) = XqPt{0)xo. Let us now consider the "reverse time" sequence 

of matrices defined as Xt = ^;(0). Since Prit) = /'t-f (0) for all t < T, the sequence {X^j^^fsj is obtained by 
iterating the generalised Riccati difference equation forward with initial condition Xq = 0. The sequence 
{Jf{xo) = XQXfXo}t^^ is obviously monotonically non-decreasing (it is the sequence of optimal costs 
over intervals of increasing lengths t). Hence, the sequences {XfjfeN^ and {R + B'^XtB}t^fq are monoton- 
ically non-decreasing sequences of positive semidefinite matrices. We now show that these sequences are 
bounded. Assume, by contradiction, that limj^+oo = +0°. The sequence {X/ = j^j/eN is bounded. 

Thus, there exists a converging sub-sequence {X^J}. Let X^ be its limit. Clearly \\X^\\ = 1: let Xq G M" 
be such that \\xq\\ = 1 and {xqYX^Xq = 1. Since we assumed that for any xq there exists a trajectory that 



4 



renders 7 defined in ^ finite, we have that there exist a constant otq and an input trajectory such that 
Jf'ii^o) ^ -^(^0' — "^Oj where the first inequality follows from the optimality of the cost J*.{xq) and the 
fact that, for a given m\ the index dll) is a sum of infinite non-negative terms which is greater than or equal 



to the sum of the first ti terms of the sum. On the other hand we have J*.{xq) 
which is a contradiction. 

Since is non-decreasing and bounded, it admits limit X for t 0°. Then, limt^^Xj = 

limj_v.ocX/+i = lim;^c»R[Xf] = X. Thus, if lim/^c>oR[X/] = R[^], then R[X] = X, i.e. X is a positive 
semidefinite solution of CGDARE(E). To prove that this is indeed the case, it is sufficient to show that 
lim/^ooi?^ = R^y- In fact, the pseudo-inverse is the only possible source of discontinuity in the Riccati 



iteration. To prove the latter equality, consider the sequence {R + B^ XtB}t(^fq. Since it is a monotonically 
non-decreasing sequence of positive semidefinite matrices, the chain of inclusions 

ker(7?+5%5) D ker{R+B^XiB) D ker(7?+5%5) D . . . 

holds. Clearly, there exist a f such that for any t >t this chain becomes stationary, i.e., for any t>t there 
holds ker{R + B^ Xt B) = ker{R + B^ Xt+i B) . This implies that a change of coordinates independent of 
t exists such that in the new basis Rx, = R + B^XtB = diag{7?P, O}, where }f>f, is a non-decreasing 
sequence of positive definite matrices. Clearly, limt^ooRx, = Rx, so that, in this basis, i?^ has the form 
i?^ = R + B^XB = diag{i?^,0}, where R^ = limt^ooR^. Moreover, since the sequence {R^} is non- 
decreasing, R^ is also nonsingular, so that (i??)^^ (R^)^^ . Thus, in the chosen basis we have indeed 



R^^ = {R+B^XfB) 





' 




' ' 














Rl 



(2). Let 



J°{xo) = infy(.x:o,M). 



(15) 



Clearly, J°{xo) > J*ixo) = XqXiXq. Then, by taking the limit, we get J°{xo) > XqXxq. We now show 
that the time-invariant feedback control = —K^Xt yields the cost XqXxq, which is therefore the optimal 
value of the cost. Consider the cost index Jjji = Jj -\-XjXxt, where Jj is defined in (fT3l) . It follows from 
[fT6i Section II], see also [10], that an optimal control for this index is given by the time-invariant feedback 
M* = —Kj^Xt and the optimal cost does not depend on the length T of the time interval and is given by 
Jj ^ = XqXxq. Notice that for this conclusion we only need the fact that X is a positive semi-definite 
solution of CGDARE(E). Now we have 



XqXxq < J°{xq) < J{xq,u*) = Y^[xJ 



,*N T 



f=0 



Xt 



iim£[x7 K*)^]n 



f=0 



Xt 

u. 



= lim J* ^ —XjXxt < lim XqXxq = XqXxq. 

T— S-OO ' T— 7-00 



(16) 



Comparing the first and last term of the latter expression we see that all the inequalities are indeed 
equalities, so that the infimum in (fTSl) is a minimum and its value is indeed XqXxq. 

(3). Suppose by contradiction that there exist another positive semidefinite solution X of CGDARE(E) 
and a vector G M" such that XqXxq < XqXxq. Take the time-invariant feedback Ut = —K^Xt. The same 
argument that led to (fT6l) now gives J{xq, u) < XqXxq < XqXxq, which is a contradiction because we have 
shown that XqXxq is the optimal value of the cost function J. 



5 



(4). Let be the set of optimal control inputs at time t = 0. Let uq G M'" and xi = Axo + Buohe the 
corresponding state at ? = 1 . Clearly the optimal cost can be written as 



' X 




Xo 







Uq _ 



r =XqXxo = [Xq Uq ] 

Moreover, uq G '^q if and only if the optimal cost can be written in the following alternative form: 

7* = xjxxi + [x^ "o ] n 



Xo 



X[A 5 ] +n 



Xq 
Uq 



By subtracting the first expression from the second, we get that uq G 



if and only if 



[ -^0 "o ] 

lent to T „ 

because the co 
(El). 



Qx Sx 



[mo ] ~ ^' ^^^'^^ ^i^'l hence TI^ 



Qx Sx 
sj % 



, are positive semidefinite, this is equiva- 



[ "0 ] ^ ^' Fiii^Uy' this is equivalent to uq = —7?^ xq + vq, where vq G M'" is arbitrary, 
umns of Gx form a basis for kerR^. By iterating this argument for all ? = 1 , 2, . . ., we get 



3 Preliminary technical results 

In this section, we present several technical results that will be used in the sequel. Most of these are 
ancillary results on the Stein equation and on spectral factorisation of independent interest. 

3.1 The Hermitian Stein equation 

In this section, we give some important results on the solutions X of the so-called Hermitian Stein 
equation (known also as the discrete-time Lyapunov equation): 



X=A^XA + Q, 



(17) 



where A, 2 G and Q = Q'^>0. 



Lemma 3.1 Let X be a solution of the Hermitian Stein equation ([73). Then, kerX is A-invariant and is 
contained in the null-space ofQ. 

Proof: Let A G C be on the unit circle and such that (A + A /„) is invertible. We can re- write (fTTl) as 

X=A^X{A + Xl„)-?iA^X + Q, (18) 

so that 

(AAV/,0X = A^X{A + Xln)+Q = XA^X{X*A+In)+Q, 
since A is on the unit circle (which means in particular that A* = A^). This is equivalent to 

X{X*A+In)-^ 



A ( AA T+4) - U + ( AA ^+7,0 - ' Q (A *A+/„) - ' . 



(19) 
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Let E, E kerX. On pre-multiplying (fT9l ) by i^* and post-multiplying it by ^, we obtain (^*(AA^ + 
In)~^Q{^*A + Iny^^ =0, and since (AA^ + /„) ^ 2 (A*A +/„) ^ is Hermitian and positive semidefi- 
nite, we get 

Q{X*A + Q-^^=0. (20) 

Let us now post-multiply (fT9l) by the same vector . We get X {X*A + In)^^ ^ =0, which means that 
kerX is (A*A -invariant. Hence, it is also ( A* A + /„) -invariant and therefore A-invariant. In view 
of (l20l) . kerX = (A*A -|-/„)^^ kerX is also contained in the null-space of Q. ■ 
We recall that equation (flTl) has a unique solution if and only if A is unmixed, i.e. for all pairs 
Ai , A2 G O'(A) we have Ai A2 7^ 1 . In this case we have the following result. 

Lemma 3.2 Let A be unmixed and X be the unique solution of 4771) where Q = > 0. Then, kerX is 
the unobservable subspace of the pair (A, Q). 



Proof: Let the pair (A, Q) be written in the Kalman observability canonical form, i.e., A 



All O 
All ^22 



and 



Q 



Qi O 
o o 



, where the pair (An, Qi) is completely observable. Let us write (flTI) in this basis. We find 



^11 ^12 
^12 ^22 



^11 
O 



^21 

A'^ 
^22 



^12 



A^ 



22X22A22 



+ 



^12 
^22 

Qi 
o 



An 
All 

O 
O 



o 

A22 



4- 



Qi 
o 



o 
o 



Therefore, X22 satisfies the homogeneous Hermitian Stein equation X22 = Aj2^22A22- Since A is unmixed, 
the submatrix A22 is unmixed, and therefore X22 = is the unique solution of X22 = Aj2X22^22- As such, 
the Hermitian Stein equation in this basis can be simplified as 



"^11 


X\2 




-k Al{XnA22 


Aj2 







_A2W2^ii 



+ 



Qi O 
o o 



Again, since A is unmixed, for all X\ G o{A\\) and A2 G cr(A22) we have that A1A2 7^ 1, and the top-right 
block of the latter equation yields the unique solution X12 = 0. Therefore, we get the following equation 
for Xii: Xii = Aj^XiiAii + Qi. In view of Lemma IBTTI the unique solution Xn of the latter equation 

where the 



im 



has trivial kernel because the pair (An, 2i) is observable. This implies that kerX 
partition is consistent with the block structure of X. On the other hand, this subspace is exactly the 
unobservable subspace of the pair (A, Q). ■ 

Remark 3.1 When the solution X of the Hermitian Stein equation (flTl) is not unique, by Lemma [BTTI the 
null-space of X is still A-invariant and is contained in the null-space of Q, but it could be strictly contained 
into the unobservable subspace of the pair (A, Q) - which we recall is the largest A-invariant subspace 
contained in the null-space of Q - without necessarily being equal to it. Moreover, it is possible that 
none of the solutions of the Hermitian Stein equation are such that kerX coincides with the unobservable 

subspace of the pair (A, 2). Consider for example the Hermitian Stein equation (flTl) with A 



1 
1 1 



and 



Q = 
is 



1 




. In this case, it is easy to see that the set of all solutions of the Hermitian Stein equation (fTTl) 
. The null-space of any solution of the Hermitian Stein equation is zero. 



a —J 






: aGKj 







Hence, none of the solutions of the Hermitian Stein equation is such that its kernel is equal to im 
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The last result we need is the following. 



Lemma 3.3 Let A e 



\F e 



\ B e 



and assume X eW^" is such that 
XF - 



X 
O 



(21) 



Then, B^{A^fX 
{A\B^). 



for all k > 0, i.e., imX is contained in the unobservable subspace of the pair 



Proof: We first prove that B^X = 0. Let us choose a basis in which F is written as F = diag{A^,i7}, 
where is nilpotent and Fj is invertible. Let us decompose X accordingly, i.e., X = [ Xi X2 ] . It is very 
easy to see that A N = Xi implies Xi = 0. In fact, by multiplying such equation by A ^ and N to the left 
and to the right, respectively, we obtain Xi = (A^)^Xi A'^^ for all ^ > 0. By choosing k to be greater than 
the nilpotency index of A'^, we get Xi = (A^)*^Xi A'^*^ = 0. From (|2T]) we also obtain B^XiFi = 0, which 
implies 5^X2 = since Fj is invertible. Therefore, B^X = 0. 

The same argument can be iterated to prove that 5^ (A^)^X = for all ^ > 0. Indeed, by pre-multiplying 
the first of (|2T]) by we get A^(A^X)F = A^X. By pre-multiplying the same equation by 5^, we get 
5^(A^X)F = since we already proved that B^X is zero. Hence, we can write these two equations as 



(ATX)F: 



o 



and re-apply the same argument used above to show that B^ A^X = 0, and so on. 



3.2 Spectral Factorisation 



Since as aforementioned the Popov matrix IT is assumed symmetric and positive semidefinite, we can 
consider a factorisation of the form 



n 



' Q 


s ' 








R 







[C D] 



(22) 



where Q = C^C, S = C^D and R = D^D. Let us define the rational matrix W{z) =C{zIn -A)^B + D. 
The spectrum 4>(z) = W [z) W{z) - where W [z) = V7^(z^^) - associated with the Popov triple E can 
be written as 



^{z)=[B^Z-'ln-AT' In ] 



Q s 

R 



{zIn-A)-'B 

In 



which is also referred to as Popov function associated with GDARE(E), The matrix inequality for an 
unknown matrix X = X^ of the form Ux > is called the discrete Riccati linear matrix inequality, and 
is herein denoted by DRLMI(E). Let us also define 



L{X)=Tix-n 



A^XA-X A^XB 
B^XA B^XB 



Notice that L{X) is a linear function of X. 

Lemma 3.4 ([18, p.322], see e.g. (21 for a detailed proof). For anyX=X^e M"^", there holds 



4.(z)=[B^(z-i/„-A0-i /„]n 



X 



{zIn-A)-'B 



(23) 
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Theorem 3.1 Let r denote the normal rank of the spectrum 4>(z)|3 IfX is a solution of CGDARE(ll), the 
rank ofRx is equal to r. IfX is a solution of DRLMI(L), the rank ofRx is at most equal to r. 

Proof: Let us consider X =X'^ such that > 0. By Lemma [2?T1 this means in particular that (i) Rx 
is positive semidefinite, (ii) ker5;c 3 k&xRx, and (iii) Qx — SxR]^S]^ is positive semidefinite. Notice that 
(iii) means that X satisfies the discrete Riccati inequality 

^{X)'^A^XA 
-{A^XB + S){R + B^XB)\B^XA + S^)+Q-X > 0. 

Therefore, we can write ^(X) = H^Hx for some matrix Hx, which leads to the expression 



n 



X 



SxRI 



Rx[Ris^x n+ 



HI 
O 



[Hx O] 



(24) 



By plugging (l24l) into (1231) we see that the spectrum 4>(z) =W {z)W{z) can be written as 

= {z)Wi{z) +W2 {z)W2{z) 

Wi{z) 



where Wi (z) is given by 



[ w;{z) w;iz) ] 



W2iz) 



and ^2(2) is given by 



Wi{z) = Rl[RlsI I] 



Rl{Rlsl{zIn-A)-'B + I, 



{zIn-A)-^B 

In 

-1 ; 



W2{z) =[Hx O] 



{zIn-A)'^B 
L 



Hxizln-A) 'B 



clef i 

Notice that Wi{z) = R\Tx{z), where Tx{z) = Rx^x {zin —A)^^B-\-Im is square and invertible for all but 
finitely many z G C. Its inverse can be written as T^^{z) = Im — RxS^izhi —Ax)^^B. Thus, the normal 
rank of 



mz)Tx\z 



is equal to the normal rank r of ^(z). Then, the rank of R^, which equals that of Rx, is not greater than 
r. Now consider the case where X = is a solution of CGDARE(E). In this case, the term Hx in (l24l) 
is zero, and therefore so is the rational function W2{z)- As such, W\ (z) is a square spectral factor of ^(z), 
i.e., W ~ (z) W^(z) = (z) Wi (z) . Moreover, (z)4>(z) T^\z) = Rx, which implies that when X=X^ 
is a solution of CGDARE(E), the rank of Rx is exactly r. ■ 



'The normal rank of a rational matrix M(z) is defined as normrankM(z) ^ maXj.gcrankM(z). The rank of M{z) is equal 
to its normal rank for all but finitely many z G C. 
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4 Geometric properties of the solutions of GDARE 



The first aim of tliis section is to show that, given a solution X of GDARE(E) 

• the subspace kerX is an output-nulling subspace for the quadruple (A,5,C,Z)), i.e., 



A 
C 



kerX C (kerX © Op) + im 



B 
D 



(25) 



the gain Kx is such that —Kx is a friend of kerX, i.e., 

kerZ CkerZeO. 



A-BKx 
C-DKk 



(26) 



In the case where X =X^ h the solution of GDARE(E) corresponding to the optimal cost, these properties 
are intuitive. Indeed, on the basis of the optimality and of the fact that the cost cannot be smaller than 
zero in view of the positivity of the index, it is not not difficult to prove that the following stronger result 
holds. 

Proposition 4.1 Let X be the minimal positive semidefinite solution of GDARE(L). Then kerX is the 
largest output-nulling subspace of the quadruple (A, 5, C, D). Moreover, —Kx is the corresponding friend. 

Proof: Let xq G kerX. Since the corresponding optimal cost is 7 = XqXxq = 0, the initial state xq must 
belong to the largest output-nulling subspace of the quadruple (A,5,C,Z)). Vice-versa, if we take a vector 
Xq of the largest output-nulling subspace of the quadruple (A,5,C,D), by definition it is possible to 
find a control u^ (k > 0) such that the state trajectory lies on by maintaining the output at zero. This 
means that the corresponding value of the cost is zero. Hence, XqXxq = implies xq E kerX. The fact that 
—Kx is a friend of kerX follows straightforwardly from the fact that if the initial state of the system lies 
on kerX and we assume by contradiction that (A —BKx)xo ^ kerX, then the corresponding trajectory is 
not optimal because it is associated with a strictly positive value of the cost. Moreover, since the optimal 
cost is zero, we must have (C — D^x) kerX = Op. ■ 
These ideas can be easily generalised to prove (l25l) and (l26l) for any positive semidefinite solution 
X = > of GDARE(E). Our aim is to prove a deeper geometric result: (l25l) and (l26l) hold for any 
symmetric solution X of GDARE(E). 

Theorem 4.1 Let X be a solution of GDARE(L). Then, kerX is an output-nulling subspace of the 
quadruple {A,B,C,D) and —Kx is a friend o/kerX, i.e., ( [25\l and i{26t) hold. 



Proof: Since X is a solution of GDARE(E), the identity 

X=A^XAx + Qox 



(27) 



holds, where Qqx = [/„ -5x4] 



Q s 

S'^ R 



I,, 



> 0. In view of Lemma [37T1 kerX is A^-invariant and is 



contained in the null-space of Qqx- By factorising IT as in (l22l) . we get Qqx = C^Cx where 



Cx=[C D] 



C-DR]rSl. 



(28) 



Hence, the subspace kerX is also contained in the null-space of Cx so that kerX is output-nulling for the 
quadruple (A,5,C,D) and —Kx is a friend of kerX. ■ 
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Our aim at this point is to provide a full characterisation of the reachable subspace on kerX, for reasons 
that will become clear in the sequel. Indeed, we will show that this subspace plays a crucial role in the 
solution of the associated optimal control problem. We recall the following definition. 

Definition 4.1 The reachable subspace on an output-nulling subspace 'Y is the subspace of the 
points ofY that can be reached from the origin along trajectories contained on Y by at the same time 
maintaining the output at zero. 

We will show that the reachable subspace -^^grX kerX, coincides with the classic reachable sub- 
space from the origin of the pair {Ax^BGx)- In order to prove this fact, we first need to show some 
important additional results on the solutions of CGDARE(E). In particular, we now focus our attention 
on the term Rx = R + B^XB. We see immediately that when X is positive semidefinite, the null-space of 
Rx is given by the intersection of the null-space of R with that ofXB. This result, which is very intuitive 
and easy to prove for positive semidefinite solutions of CGDARE(E), indeed holds for any solution X. 
However, in this case the proof - which is divided between Lemma HTTI and Lemma l4~2] presented below 
- is much more involved, and requires the machinery constructed in the first part of the paper. 

Lemma 4.1 Let X =X^ be a solution of CGDARE(L), Cx be defined by d2gl) and 

^o'^im[ BGx AxBGx A^BGx ... A^^^BGx]- (29) 

Then, 

ker%Ckeri?, and ^o^kerC^. (30) 

Proof: Since the columns of Gx (defined in ([8])) span kerRx, we need to show that RGx = 0. Recall 
from the proof of Theorem 13. II that when X = X^ is a. solution of CGDARE(E), the spectrum 4>(z), can 
be written as 4>(z) = W^{z}W{z) = T^ {z)RxTx{z) where Tx{z) = R\sl{zln -A)-^B+I^ is square 
and invertible for all but finitely many z G C. Hence, we have Rx — [W{z)T^^{z)] [W{z)T^^{z)] so 
that RxGx = implies W (z) T^ \z)Gx = 0. Now recall from the proof of Theorem [37l] that T^ ^ (z) = 
Im —Rx^xizln —Ax)^^B SO that we can compute 

W{z)T^'{z) = {C{zI„-A)-'B+D) {lrr,-Rlsl{zIn-Ax)-'B^ . 

Consider the following term of the product: 

H{z) =C{zIn -A)-'BRlsl{zIn -Ax)-'B. 

By noticing that BR^S]^ = A -Ax = (z4 - A^) - (z4 -A), we obtain 

H{z)=C{zI„-A)-^B-C {zin -Ax)-^B. 

Hence, 

W{z)Tx\z) = W{z)-DRlsl{zIn-Axy^B-H{z) 
= D + {C~DRlsl){zIn-Ax)-'B 
= D + Cx{zln-Ax)-^B. 

Since W{z)T^^ (z) Gx is identically zero, it must be zero also when z — )■ oo. In particular, DGx = 0, so that 
RGx = 0, which yields the first of From W{z)Tx\z)Gx = we also get CxizI -Ax^^BGx =0so 
that the reachable subspace of the pair (Ax^BGx), i.e. (|29]) , is contained in kerCx so that also the second 
of (1301) holds. ■ 
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In Lemma \4~T\ we have shown that kerRx C keri?. Since Rx — R+B^X B, it also straightforwardly 
follows that kevRx C kcr{B^XB) for any solution X of CGDARE(E). However, a stronger result holds, 
which says that kevRx C ker(X5). This is an obvious consequence of Lemma HTTI for any solution 
X > 0, while it is a quite surprising and deep geometric result in the general case. 



Lemma 4.2 Let X = X^ be a solution of CGDARE(L). Then, 

ker7?jf Cker(X5). 



(31) 



Proof: From Lemma WA\ if v G keri?^^ then v G kerT? fl ker(5^X5). We can select a change of 
coordinates in the input space M"' induced by the mx m orthogonal matrix Tx — [ Tix T2X ] where 
imTix = imRx and imr2x = imG^ = kerRx- In this basis Rx is block-diagonal, with the first block 
being non-singular and the second being zero. Since kerR 3 kerRx as proved in Lemma 14.11 matrix 

R in this basis has the form R = ^ . In the same basis, matrix B can be partitioned accordingly as 

B=[Bi 52 ] , so that im52 = im{BGx). We must show that X52 = 0. Since ker% C ker:{B'^XB), in 
this basis we find 



B^XB2 



Bl 



XB2 = 0. 



(32) 



Moreover, since kerR C ker5, in the selected basis S takes the form S = [ Si ] . Thus, Sx =A^XB + 
S= [A'^XBi+Si A'^XB2 ]. From ker% C ker 5x it now follows that A "^X 52 = which, together with 
(|32l). yields 

^52 = 0. (33) 

If A is non-singular or, more in general, if the zero eigenvalue of A, when present, is controllable from B, 
then clearly X52 = 0. However, this result is true in general, without any assumption. To prove this, let us 
consider Mq defined in (|29l ) which, in the chosen input space basis, is the reachable subspace of the pair 
(Aa:,52)J1 Let us consider a basis of the state-space where the pair (Ax,52) are in Kalman controllability 

form. In such a basis, the subspace ^0 is spanned by the columns of the matrix 



and we have 



Ax 



Ax,ll Ax,12 

O Ax72 



B2 



B21 
O 



Bi 



Bn 
B12 



(34) 



where the pair (Ax,ii,52i) is reachable. In this basis, matrix Cx takes the form Cx = [0\ Cx,\\ in view of 

XB2 = or, equivalently. 



the second of (1301) . Since Ax = A — BKx, we can re-write (1331) as 



as 



XB2 = 0. Using the partitioned structure described above, we can re- write this equation as 



0. 








4T 


4T 


^11 


^12 


^21 






Xil Xi2 




B2I 


_ ^12 ^22 _ 








(35) 



We want to show that 



'^11 X12' 




B21 


_X^2 ^22. 




_ _ 



0, i.e., that Xi 1 ^21 = and Xj^2^2i = 0. From ([35]) we find 
0, 



^1,11^11^21 
-Sjl ^11-^21 



0. 



(36) 
(37) 



^In the symbol denoting this subspace we dropped the subscript X because, as it will be proved in the sequel, this subspace 
is independent of the particular solution of the CGDARE(E). 
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Since the pair (Ax, 1 1 , ^21 ) is reachable by construction, Xi 1 ^21 =0. It remains to show that ^^2^21 = 0. 
In this basis, equation (|27l) - which is exactly GDARE(E) - now reads as 



' Xn 


X\2 




. ^12 


X22 _ 





O 



^XA2 ^X,22 



Xn X\2 

42 



X,o X22 



Ax,ll Ax, 12 

O Ax22 



+ 



^x.i 



[ Cx,i ] 



(38) 



from which we find in particular Xn = iiXiiAx,ii. This equation can be written together with (|36l) 
and (137]) as 

^X.ll^ll'^^,!! "^X, 11^11-^21 

5jjXiiAx,ii 5jjXii52i 

Since the pair (Ax.ii,52i) is reachable, we can apply Lemma 2.9 in [18], which guarantees that Xn is 
zero. Now we can re- write (|38]) as 



0. 



' X12 " 




_ ^12 ^22 _ 





^X,ll 



O 



+ 



^X,\2 ^X,22 

o o 



O X12 

42 



Xio X22 



Ax, 11 Ax, 12 

O Ax,22 



In particular, we get 

^12 = Ax,iiXi2Ax,22- 

Moreover, by plugging the value Xn = into (|35]) . after transposition, we obtain 

5jiXi2Ax,22 = 0. 

Equations (|40]) and (|4T]) can be re-written as 



^X,ll 

^21 J 



^12Ax,22 



^12 

o 



(39) 
(40) 
(41) 

(42) 



By applying the result in Lemma 133] for = 0, we get X12 = 0. ■ 

Remark 4.1 In the last line of the proof of Lemma |43] we can applying Lemma [33] for ^ > 0, and obtain 
^I2^x 11^21 = for all ^ > 0. Since the pair (Ax,ii,52i) is reachable, this yields X12 = 0. Therefore the 
following stronger result holds. 

Proposition 4.2 LetX=X^bea solution ofCGDARE(L) and Mq be defined by ([29]). Then, XMq = 0„. 
Remark 4.2 As an obvious corollary of Lemmas [4.11 and [43] we have that 



keri?x = ker(X5) nkeri? = ker 



XB 
R 



(43) 



Remark 4.3 The result established in Lemma [4~2[ does not continue to hold if we only assume that X = 
X^ is a solution of GDARE(E) (instead of being a solution of CGDARE(E)). Consider for example the 

, C = [0 1 ] and D = [4 0] . It can be easily verified that X = diag{ — 1 , 1 } is 



case A 



-1 

-5 -6 



B 



-4 
-2 



a solution of the GDARE(E) but not of the CGDARE(E). In this case, keri?x = im 



7^ ker 



XB 
R 



0„ 
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Remark 4.4 The result established in Lemma \4~2] does not hold when the Popov matrix is not positive 



semidefinite. Consider the following numerical example in which A 



1 1 
1 1 



B 



1 
1 



Q 



1 

-1 








and 7? 



1 




A solution of CGDARE(E) is given by X = diag{l, —1}. Indeed, X satisfies 



dH) since one can easily verify that Sx is the zero matrix and kerRx = im 



the input space as shown in the proof of Lemma l4.2l leads to Bi 
XB2 



and B2 



Changing coordinates in 
However, this time 



1 

-1 



Theorem 4.2 Let X = be a solution of CGDARE(L). Let denote the reachable subspace of the 
pair {Ax,BGx) as defined in f [29l) . and My be defined by Definition \4.1\ Then, 



KeiX 



Proof: Let us first show that 



im(5G 



kerXnfikerZ). 



(44) 



(45) 



ker(X5) n keri?. Then 



so that M^^^x '•^^ smallest 



We recall that imGx = kevRx- Moreover, from (1431) we know that kevRx 
im{BGx) = BkevRx = 5(ker(X5) nkerT?) = kerXDBkerR = kerX flfikerD. 

Now we are ready to prove the statement of this theorem. Since Mq is the reachable subspace 
from the origin of the pair (Ax.BGx), it is by definition the smallest -invariant subspace containing 
im{BGx) = kerX DBkerD. On the other hand, the reachable subspace M^^^x kerX is characterised 
as follows [|19> Theorem 7.14], [[T4l p. 424]: Let F be an arbitrary friend of kerX, i.e., F is any feedback 
matrix such that (A+BF) kerX C kerX and (C + DF) kerX = Op . Then M^^^^ is the smallest (A + BF)- 
invariant subspace containing kerX fl B kerD. Notice that M^^^x depend on the choice of the 

friend F, (19, Theorem 7.18]. We have seen in Theorem 14. II that the matrix F = —Kx is a particular 
friend of kerX. For this choice of F, we have A + BF = A— BKx = Ax 
Ax -invariant subspace containing kerX n5 kerD, which is exactly the definition of 

Remark 4.5 The statement of Theorem l4.2l can be also captured as follows. First, we know from Lemma 
14. II that Cx {zl/i —Ax ) ^ ^5 is identically zero, which implies that the reachable subspace from the origin 
of the pair (Ax.BGx) is contained in the non-observable subspace of the pair (Ax.Cx)- Thus, Mq is also a 
controllability subspace for the quadruple (Ax,0,Cx,0) and a controllability subspace for the quadruple 
(A,5,C,D). Consider the same orthogonal change of coordinates in the input space introduced in the 
proof of Lemma |4]2] induced by the m x m matrix Tx = [ Tix T2X ] • Let matrix B be partitioned in this 
basis as 5 = [ 5i ^2 ] . This change of coordinates gives rise to a state- space model of the form 

Xk+i = AxXk+[Bi B2] Vk, 
yk = CxXk+[Di 0]vk. 



By using the control = , we find that the state x^ can reach every point of the reachable subspace 

from the origin of the pair {Ax,B2) and at the same time the output y^ is kept at zero. Therefore, this 
subspace is also a controllability subspace for (Ax,0,Cx,0). 

In ifTSl it is proved that the inertia of Rx is independent of the particular solution X = X^ of 
CGDARE(E). Here, we want to show that much more is true when IT is positive semidefinite. Namely, 
the null-space of Rx is independent of the particular solution X =X^ of CGDARE(E). 
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Theorem 4.3 Let Xi,X2 be two solutions of CGDARE(L). Then, keri?Xi = keri?x2. 



Proof: Consider two solutions Xi = X^ and X2 = X2 of CGDARE(E). In particular, Xi and X2 also satisfy 
the generalised Riccati inequality, so that > and > 0. In other words, using the same notation 
employed in the proof of Theorem l3.1[ we have n^, = L{Xi) + n > for z G {1,2}. The set of solutions 
of the generalised Riccati inequality is a convex set, i.e., by taking a G (0, 1), then aUx^ + — oc) H^j is 
positive semidefinite because it is a convex combination of positive semidefinite terms. For our purposes, 
it is sufficient to fix an arbitrary value of a, say cc = j. Then, 



n+L( ^{x,+X2) 



where the last equality holds in view of the linearity of L(-). This means that y =4 (Xi +X2) satisfies the 

Riccati inequality Tly > 0. By virtue of Theorem 13. 11 the rank of Ry = R + B^Y B is not greater than r. 
On the other hand, 

Ry =R + ^B^(X, +X2)B = i {Rx, +Rx2)- (46) 

Hence, since Xi and X2 are both solutions of CGDARE(E), the ranks of Rx^ and Rx2 are exactly equal to 
r. Thus, the rank of Ry is greater or equal to r. This means that the rank of Ry must be exactly equal to 
r, i.e., from (|46|) we have that Rx^ and Rx2 must have the same null-space. ■ 
Now we want to prove that the subspace ^^^^x independent of the particular solution X = X^ of 
CGDARE(E). Moreover, Ax restricted to this subspace does not depend on the particular solution X = 
of CGDARE(E). 



Theorem 4.4 Let X and Y be two solutions of CGDARECL). Let Ax and Ay be the corresponding closed- 
loop matrices. Then, 



-kerX 



kerX 



■-Ay\ 



''kerY 



Proof: LetA = y-X. Since kerRx coincides with kerRy by virtue of Theorem 14. 3[ we have i?^ 
RlRyRl so that 



Ax-Ay = B{Rls^-RlSl)=BRl{S^-RyRlSl] 



Plugging 



and 



B^YA+S^ = B''AA+S^+B''XA = B^AA+S^ 



Ry =R + B^YB-B^XB + B^XB = Rx+B^AB 



into (1471) yields Ax — Ay 



BRliB^AA-B'^ABR^S^) 



BRyB^AAx. This means that Ay 



(47) 

(48) 

(49) 

Ax- 



BRyB^ AAx- We already know that in a suitable basis of the state space such that the first coordinates 
span ^^gj-x and a suitable orthogonal basis of the input space such that the second group of coordinates 
span kerRx, matrices Ax and B can be written as in (l34l) . see Lemma 142] The reachable subspace of the 



pair (AxtBGx) is written in this basis as 
have 

" AxM 
O 



. We want to show that, in the same basis, we also 



AyA2 
Ay 72 



and B2 



B21 
O 



(50) 
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In fact, if this is the case, the reachable subspace of the pair (Ay,5Gy) exactly coincides with the 
reachable subspace of the pair (Ax.BGx), i.e., with Mq, because the pair {Ay^22i0) is completely non 
controllable. In the chosen basis, the difference A = Y —X can be written as A = diag{C), A2} in view of 
[[18. Theorem 2. 10]. H Thus 



Ay 



brIb^ 



' 


' 







A2 _ 





Ax- 



Ax,n 
O 



Ax.u 
Ax, 11 



•k -k 




' 







Ax,ii * 


•k ★ 







^2 Ax, 22 




★ 



Therefore, the reachable subspace of the pair {Ay,BGy) is exactly 
Ax.n. 



--Ay\ 



We conclude this section by briefly summarizing what we have obtained so far. We have identified a 
subspace such that the closed-loop matrix restricted to this subspace is independent of the particular 
solution of CGDARE(E). This means that if this part of the spectrum contains unstable eigenvalues, 
CGDARE(E) does not admit stabilising solutions for the associated optimal control problem. However, 
this subspace is a controllability subspace, so it always admits a stabilising friend. This consideration, 
together with the fact that when Mq is non-zero the optimal control is not unique and is parameterised as 
in (fT2l) . will lead to the interesting result that the closed-loop can be stabilised exploiting the additional 
term Gx in (fT2l) . even in cases in which CGDARE(E) does not admit stabilising solutions. 



5 Stabilisation 

In the previous sections, we have observed that the eigenvalues of the closed-loop matrix Ax restricted 
to the subspace ^0 are independent of the particular solution X =X^ of CGDARE(E) considered. This 
means that these eigenvalues - which, as we will show elsewhere [5], do not appear as generalised 
eigenvalues of the extended symplectic pencil - are present in the closed-loop regardless of the solution 
X = X^ of CGDARE(E) that we consider. On the other hand, we have also observed that ^0 coincides 
with the subspace ^^^erX' which is by definition the smallest (A — 5 -invariant subspace containing 
kerX DBkerD = im{BGx)- It follows that it is always possible to find a matrix L that assigns all the 
eigenvalues of the map {Ax + BGxL) restricted to the reachable subspace ^^^^x^ adding a further 
term B Gx Lx^ to the feedback control law, because this does not change the value of the cost with respect 
to the one obtained by = —Kxx^. Indeed, the additional term only affects the part of the trajectory on 
^kerX which is output-nulling. However, in doing so it may stabilise the closed-loop if kerX is externally 
stabilised by —Kx- We show this fact in the following example. 



Example 5.1 Consider a Popov triple in which A 



1 1 
1 



, B 



2 
1 1 



Q 




1 








andR 








. The matrix X = diag{0, 1 } is the only solution of GDARE(E) but not a solution of DARE(E), since 
R + B^XB is singular. Hence, DARE(E) does not admit solutions. The corresponding closed-loop matrix 



^The result in |[T8l Theorem 2.10] is shown in a basis that is the same considered here. Indeed, the basis of the state 
space considered in fTS', Theorem 2.10] has the first coordinates spanning the largest controllability subspace of the quadruple 
{A,B,Sx,Rx)- However, this subspace coincides with the largest controllability subspace of a quadruple obtained from the 
previous one by applying the control input m, = —KxXt +Hx v,, where imHx = keiRx- The quadruple thus obtained is exactly 
{Ax , B Gx ,Sx — RxRx'^X'O) — {Ax , B Gx ,0,0), and the corresponding largest controllability subspace is indeed ^^^^-x ■ 
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Ax is Ax — diag{l,0}, so that the resuhing closed-loop system is not asymptotically stable. However, 
the solution X of GDARE(E) is optimal for the LQ problem, because it leads to the cost /* = ^2(0) 
which cannot be decreased. Now, consider the gain K — B^^A. This gain leads to the closed-loop matrix 
AcL = A—BK = 0, and the value of the performance index associated with this closed-loop is again 
/ = X2(0) = /*. Therefore, this is another optimal solution of the LQ problem, which differently from X 
is also stabilising. However, this optimal solution is not associated with any solution of GDARE(E), since 
as aforementioned X is the only solution of GDARE(Z). In other words, this example shows that there 
exists an optimal control which is stabilising, but no stabilising solutions of GDARE(E) exist. This fact 
can be explained on the basis of the fact that the set of all solutions of the infinite-horizon LQ problem is 
given by 

% = {-KxXk + GxVk\ v^eM""}, 



Rx^x) - 



1 -1 
-1 1 



Therefore, the 



where X is the optimizing solution of GDARE(E) and Gx = (In 
problem becomes that of using the degree of freedom given by in order to find a closed-loop solution 
that is optimal and also stabilising. In other words, we determine a matrix L in 



Xt+i = {A-BKx)xt+BGxLxt ^AxXt +BGxLxt 



such that the closed-loop Aql 
optimal closed loop matrices Aql 



Ax +BGxL is stabilised. It is easy to see that, in general, the set of all 

a 

0. 

1 -1 ■ 



Ax+BGxLare parameterised by Aql 



where a and j8 can 







, by choosing L ■■ 



a-l p 




we 



be arbitrarily chosen by selecting a suitable L. In fact, since 5 Gx 
obtain the desired form for the closed-loop matrix. Hence, in particular, we can obtain a zero or nilpotent 
closed-loop matrix. In both cases, the cost is the same and is equal to /* = x^iO). 

In other words, there is only one solution to GDARE(E) and is not stabilising, and all the optimal 
solutions of the optimal control problem are given by the closed- loop matrix Ax + BGx L, where L is 
a degree of freedom. By using this degree of freedom, we have found solutions of the optimal control 
problem that are stabilising but which do not correspond to stabilising solutions of GDARE(L), because 
GDARE(L) does not have stabilising solutions. □ 



Concluding remarks 

In this paper we presented a self-contained analysis of some structural properties of the generalised alge- 
braic Riccati equation that arises in infinite-horizon discrete linear quadratic optimal control. Important 
side results on Hermitian Stein equations and on spectral factorisation have been established to the end 
of showing the fundamental role that the term Rx plays in the structure of the solutions of the CGDARE 
and of the corresponding LQ problem. The considerations that emerged from this analysis have in turn 
been used to show that a subspace ,^0 can be identified that is independent of the particular solution of 
CGDARE considered. Even more importantly, it has been shown that the closed- loop matrix restricted to 
this subspace does not depend on the particular solution of CGDARE. This structural property of GDARE 
can also be displayed via a decomposition on the extended symplectic pencil, which shows that the spec- 
trum of the closed-loop matrix restricted to Mq is not reflected on the generalised eigenstructure of the 
extended symplectic pencil. In other words, this part of the spectrum has been shown to be fixed for any 
state-feedback control constructed from a solution of the CGDARE. On the other hand, if such subspace 
is not zero, i.e., when the related extended symplectic pencil is not regular, in the optimal control a further 
term can be added to the state-feedback generated from the solution of the Riccati equation that does not 
modify the value of the cost. This term can in turn be expressed in state-feedback form, and acts as a 
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degree of freedom that can be employed to stabilise the closed-loop even in cases in which no stabilising 
solutions exists of the Riccati equation. 

The results presented here, — as it will shown in a forthcoming paper f5] — can be used to generalise 
the approach taken in to the case of non-regular extended symplectic pencil for the solution of 
constrained finite-horizon LQ problems. 
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